Method and apparatus for encoding and decoding video signal using adaptive sampling

ABSTRACT

Disclosed herein is a method of performing a deblocking filtering on a video signal. The method may include determining an adaptive sampling rate or an adaptive sampling pattern based on the property information of a decoded picture and performing the deblocking filtering using samples to which the adaptive sampling rate or the adaptive sampling pattern has been applied. The property information of the decoded picture may include at least one of a block size and a picture parameter.

TECHNICAL FIELD

The present invention relates to a method and an apparatus for encodingand decoding a video signal using adaptive sampling and, moreparticularly, to an adaptive sampling method in an in-loop filteringprocess.

BACKGROUND ART

Compression coding means a series of signal processing technologies forsending digitalized information through a communication line or storingdigitalized information in a form suitable for a storage medium. Media,such as video, an image, and voice, may be the subject of compressioncoding. In particular, a technology for performing compression coding onvideo is called video compression.

The next-generation video content expects to feature high spatialresolution, a high frame rate, and high dimensionality of a video scenerepresentation. The processing of such content would require asignificant increase in memory storage, a memory access rate, andprocessing power.

Accordingly, it is desirable to design a coding tool which address theseforeseen challenges and offer some solutions. In particular, in existingvideo coding methods, in the case of deblocking filtering, theestimation of filter parameters is not adaptive to picture propertiesand all pixels of the decoded picture are used in a fixed manner.Accordingly, there is a need for a more efficient sampling method in adeblocking filtering process.

DISCLOSURE Technical Problem

An object of the present invention is to propose a method of enabling acoding tool for high efficiency compression to be designed and reducingrequired computation resources.

Furthermore, the present invention proposes a locally adaptive samplingmethod in video signal coding.

Furthermore, the present invention proposes a method of applying locallyadaptive sampling to the in-loop filtering stage of a video signal.

Furthermore, the present invention proposes a method of applying anadaptive sampling rate to the samples of a decoded picture placed at ablock boundary.

Furthermore, the present invention proposes a method of determining anadaptive sampling rate based on the features (e.g., a block size and apicture parameter) of a coded picture.

The technical problems solved by the present invention are not limitedto the above technical problems and those skilled in the art mayunderstand other technical problems from the following description.

Technical Solution

The present invention provides a locally adaptive sampling method invideo signal coding.

Furthermore, the present invention provides a method of applying locallyadaptive sampling in the in-loop filtering stage of a video signal.

Furthermore, the present invention provides a method of determining anadaptive sampling rate or an adaptive sampling pattern based on theproperty information of a decoded picture.

Furthermore, the present invention provides a method of performing adeblocking filtering using samples to which the adaptive sampling rateor the adaptive sampling pattern has been applied.

Furthermore, the present invention provides a method of identicallyapplying the adaptive sampling rate or the adaptive sampling pattern toa left block and a right block if the size of the left block is the sameas that of the right block on the basis of a block boundary.

Furthermore, the present invention provides a method of differentlyapplying the adaptive sampling rate or the adaptive sampling pattern toa left block and a right block if the size of the left block isdifferent from that of the right block on the basis of a block boundary.

Furthermore, the present invention provides a method of applying theadaptive sampling rate or the adaptive sampling pattern to a blockhaving a larger size with a low frequency sampling or a reducedsampling.

Furthermore, the present invention provides a method of applying theadaptive sampling rate by scaling an offset value with a rate conversionvalue.

Furthermore, the present invention provides a method of differentlyapplying the rate conversion value to a left block and a right block onthe basis of a block boundary.

Furthermore, the present invention provides a method of additionallyadjusting the location of the samples based on a sub-rate conversionoffset value.

Advantageous Effects

The present invention can enable the design of a coding tool for highefficiency compression and can also significantly reduce requiredcomputation resources, memory requirements, a memory access bandwidth,and computation complexity by proposing a locally adaptive samplingmethod.

Furthermore, a compression tool having a higher coding gain can bedesigned by removing redundancy and noise in determining a sample value.

DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic block diagram of an encoder in which encoding isperformed on a video signal in accordance with an embodiment to whichthe present invention is applied;

FIG. 2 is a schematic block diagram of a decoder in which decoding isperformed on a video signal in accordance with an embodiment to whichthe present invention is applied;

FIG. 3 schematically illustrates location relationships between sampleswithin a block used when filtering is performed in accordance with anembodiment to which the present invention is applied;

FIG. 4 is a schematic internal block diagram of a filtering unit forperforming locally adaptive sampling in accordance with an embodiment towhich the present invention is applied;

FIGS. 5 and 6 are embodiments to which the present invention is applied,wherein FIG. 5 is a flowchart illustrating a method of performinglocally adaptive sampling and FIG. 6 is a flowchart illustrating amethod of performing filtering using locally adaptive sampling.

FIGS. 7 to 9 are diagrams illustrating that an adaptive sampling rate isdetermined based on the property information of a decoded picture inaccordance with embodiments to which the present invention is applied;

FIG. 10 is a diagram illustrating a method of performing filtering usinglocally adaptive sampling in accordance with an embodiment to which thepresent invention is applied; and

FIG. 11 illustrates various examples in which a sampling is performedusing a rate conversion value and a sub-rate conversion offset value inaccordance with an embodiment to which the present invention is applied.

BEST MODE

The present invention provides a method of performing a deblockingfiltering on a video signal, including adaptively determining anadaptive sampling rate or an adaptive sampling pattern based on theproperty information of a decoded picture and performing filtering usinga pixel to which the adaptive sampling rate or the adaptive samplingpattern has been applied, wherein the property information of thedecoded picture includes at least one of a block size and a pictureparameter.

Furthermore, in an embodiment of the present invention, if the size of aleft block is identical with the size of a right block based on a blockboundary, the adaptive sampling rate or the adaptive sampling pattern isidentically applied to the left block and the right block.

Furthermore, in an embodiment of the present invention, if the size of aleft block is different from the size of a right block based on a blockboundary, the adaptive sampling rate or the adaptive sampling pattern isdifferently applied to the left block and the right block, and lowfrequency sampling or reduced sampling is applied to a block having alarger size.

Furthermore, in an embodiment of the present invention, the pixel isaddressed by a relative location for a left-most pixel of a right blockbased on a block boundary.

Furthermore, in an embodiment of the present invention, the adaptivesampling rate is applied by scaling an offset value with a rateconversion value.

Furthermore, in an embodiment of the present invention, the rateconversion value is differently applied to a left block and the rightblock based on the block boundary.

Furthermore, in an embodiment of the present invention, the location ofthe pixel is additionally adjusted based on a sub-rate conversion offsetvalue.

Furthermore, in an embodiment of the present invention, a step ofperforming the deblocking filtering includes determining a pixel to befiltered based on the adaptive sampling rate or the adaptive samplingpattern, calculating an offset value by applying a finite impulseresponse (FIR) filter to the pixel, and obtaining the filtered pixelvalue by applying a clipping function to the offset value.

Furthermore, the present invention provides an apparatus for performinga deblocking filtering on a video signal, including a picture propertychecking unit configured to check the property information of a decodedpicture, an adaptive sampling unit configured to determine an adaptivesampling rate or an adaptive sampling pattern based on the propertyinformation of the decoded picture, and a filtering execution unitconfigured to perform a deblocking filtering using a pixel to which theadaptive sampling rate or the adaptive sampling pattern has beenapplied, wherein the property information of the decoded pictureincludes at least one of a block size and a picture parameter.

Furthermore, in an embodiment of the present invention, the filteringexecution unit is further configured to determine a pixel to be filteredbased on the adaptive sampling rate or the adaptive sampling pattern,calculate an offset value by applying a finite impulse response (FIR)filter to the pixel, and obtain the filtered pixel value by applying aclipping function to the offset value.

MODE FOR INVENTION

Hereinafter, exemplary elements and operations in accordance withembodiments of the present invention are described with reference to theaccompanying drawings. The elements and operations of the presentinvention that are described with reference to the drawings illustrateonly embodiments, which do not limit the technical spirit of the presentinvention and core constructions and operations thereof.

Furthermore, terms used in this specification are common terms that arenow widely used, but in special cases, terms randomly selected by theapplicant are used. In such a case, the meaning of a corresponding termis clearly described in the detailed description of a correspondingpart. Accordingly, it is to be noted that the present invention shouldnot be construed as being based on only the name of a term used in acorresponding description of this specification and that the presentinvention should be construed by checking even the meaning of acorresponding term.

Furthermore, terms used in this specification are common terms selectedto describe the invention, but may be replaced with other terms for moreappropriate analyses if other terms having similar meanings are present.For example, a signal, data, a sample, a picture, a frame, and a blockmay be properly replaced and interpreted in each coding process.

The present invention proposes a method using locally adaptive samplingin video coding. More specifically, the present invention proposes amethod of applying locally adaptive sampling in the in-loop filteringstage in a transform-based hybrid video coding structure.

In accordance with an embodiment of the present invention, an adaptivesampling rate may be applied to the samples of a decoded picture placedat a block boundary, and the adaptive sampling rate may be determinedfrom the property information (e.g., a block size and a pictureparameter) of a coded picture.

Accordingly, the present invention can enable the design of a codingtool for high efficiency compression and can also reduce requiredcomputation resources (e.g., memory requirements, a memory accessbandwidth, and computation complexity).

An example of adaptive sampling in video coding is as follows.

Skip mode may be interpreted as locally adaptive sampling in a timedirection. Furthermore, in video coding, block partitioning may beinterpreted as adaptive sampling for coding parameters, motioninformation, and coding mode.

Furthermore, when deblocking filtering is performed, somedecision-making processes are performed at a block level withoutaccessing sampling data. This may be interpreted as locally adaptivesampling for using coding parameters (e.g., motion information, codingmode, and a transform size) of a coded picture.

Furthermore, an adaptive resolution change at a sequence level may alsobe interpreted as locally adaptive sampling. In some applications, avideo encoder may select a resolution for representing video data whichprovides an optimal Rate-Distortion (RD) cost and performs video signalresampling prior to coding. For example, a side information signalingmechanism (e.g., SEI messages) may be utilized to specify decoderoperations for resampling decoded video to the original resolution.

Furthermore, an adaptive resolution change at a picture level may alsobe interpreted as locally adaptive sampling. In some applications, avideo encoder may select a resolution for representing a coded picturewhich provides an optimal RD cost and performs video signal resamplingprior to coding. In such systems, a decoder would perform normativeresampling/resolution normalization for every specific picture beforeplacing it in a DPB unit or in a reference picture list. Scalable videocoding systems may be examples of such systems. The next-generationvideo content is likely to feature high spatial-resolution and highdimensionality of scene representation. The processing of such videocontent would require a significant increase in memory storage, a memoryaccess rate, and processing power.

In contrast, such a high sampling rate may lead to a problem of theoversampling of certain content. For example, video may be representedwith a lower sampling rate without a loss in the quality ofreconstructed data. Moreover, the application of complex models to verysimple signals may lead to a noise component, processing artifacts, anda loss of compression efficiency.

Accordingly, in order to allow the next-generation video applications tohave a reasonable computation cost, video coding systems may be designedto use multi-rate signal processing.

For example, a multi-rate signal processing method may use sampling rateconversion. The method may be applied to systems having different inputand output sample rates.

The sampling rate conversion may be used to solve a complicatedmulti-variable optimization problem, but in video signals, such anapproach may be not desirable due to a spatio-temporally varyingcomplexity.

Accordingly, the use of locally adaptive sampling rates when videocontent is processed may allow for a significant reduction in requiredcomputational resources and lead to the design of more efficientcompression tools.

For example, the adaptive partitioning of a coded picture may be used innon-overlapping picture fragments (or blocks). In such adaptivepartitioning, a basis analysis of coded video data is performed and thesampling of coding mode is selected. For example, large block sizes maybe used to code low complexity signals, and small block sizes may beused to code high complexity signals.

Furthermore, in most video coding tools (e.g., MC prediction, transform,and signaling), partitioning of other block sizes may be defined. Theblocks may be accompanied by respective coding parameters (i.e.,parameters of motion model, such as motion vector my and index ofreference picture used for prediction, refidx).

However, most coding tools, including interpolation, in-loop filtering(e.g., deblocking and a Sample-Adaptive Offset (SAO), andintra-prediction mode, are not affected by a selected partition size andthus do not benefit from the available estimates of local signalcomplexity of a coded signal.

Accordingly, the present invention proposes a method using locallyadaptive sampling in video coding. More specifically, embodiments inwhich locally adaptive sampling is applied in an in-loop filtering stageare described below.

FIG. 1 is a schematic block diagram of an encoder in which encoding isperformed on a video signal in accordance with an embodiment to whichthe present invention is applied.

Referring to FIG. 1, the encoder 100 includes a transform unit 120, aquantization unit 125, a dequantization unit 130, an inverse transformunit 135, a filtering unit 140, a Decoded Picture Buffer (DPB) unit 150,an inter-prediction unit 160, an intra-prediction unit 165, and anentropy encoding unit 170.

The encoder 100 receives an input video signal and generates a residualsignal by subtracting a prediction signal, output by theinter-prediction unit 160 or the intra-prediction unit 165, from theinput video signal. The generated residual signal is sent to thetransform unit 120, and the transform unit 120 generates a transformcoefficient by applying a transform scheme to the residual signal.

The quantization unit 125 quantizes the generated transform coefficientand sends the quantized coefficient to the entropy encoding unit 170.The entropy encoding unit 170 performs entropy coding on the quantizedsignal and outputs the resulting signal.

The quantized signal output by the quantization unit 120 may be used togenerate the prediction signal. For example, the residual signal may bereconstructed by applying dequantization and inverse transform to thequantized signal through the dequantization unit 130 and the inversetransform unit 135 within the loop. A reconstructed signal may begenerated by adding the reconstructed residual signal to the predictionsignal output by the inter-prediction unit 160 or the intra-predictionunit 165.

Meanwhile, artifacts in which a block boundary appears may be generatedbecause neighboring blocks are quantized by different quantizationparameters in such a compression process. Such a phenomenon is calledblocking artifacts, which is one of important factors on which peopleevaluate picture quality. In order to reduce such artifacts, a filteringprocess may be performed. Such blocking artifacts can be removed and anerror of a current frame can be reduced through such a filteringprocess, thereby being capable of improving picture quality.

The filtering chiefly includes in-loop filtering and post filtering. Thepost filtering may be optionally used by a display device or a userwithout affecting a video decoding process, and thus a detaileddescription thereof is omitted. The in-loop filtering is described inconnection with the filtering unit 140. The filtering unit 140 mayperform deblocking filtering or sample-adaptive offset filtering or mayperform both the deblocking filtering and the sample-adaptive offsetfiltering.

The filtering unit 140 applies filtering to the reconstructed signal andoutputs the filtered signal to a playback device or sends the filteredsignal to the DPB unit 150. The filtered signal sent to the DPB unit 150may be used as a reference frame in the inter-prediction unit 160. Bothpicture quality and coding efficiency can be improved using the filteredframe as a reference frame in inter-prediction mode as described above.In this case, a filtering computation process is complicated, andfrequent memory access is caused because the filtering unit 140 has toload reconstructed samples stored in memory and to store the filteredsamples in the memory again in order to perform filtering. As a result,the complexity of a decoder is increased. Accordingly, in an embodimentof the present invention, a coding tool capable of reducing thecomplexity of the decoder is designed. More specifically, the presentinvention may lead to reduction in required computation resources byapplying locally adaptive sampling in the filtering process.

The DPB unit 150 may store the filtered frame in order to use thefiltered frame as a reference frame in the inter-prediction unit 160.

The inter-prediction unit 160 performs temporal prediction and/orspatial prediction with reference to a reconstructed picture in order toremove temporal redundancy and/or spatial redundancy. Theintra-prediction unit 165 predicts a current block with reference tosamples around a block on which coding is to be performed.

The prediction signal generated by the inter-prediction unit 160 or theintra-prediction unit 165 may be used to generate the reconstructedsignal or to generate the residual signal.

FIG. 2 is a schematic block diagram of a decoder in which decoding isperformed on a video signal in accordance with an embodiment to whichthe present invention is applied.

Referring to FIG. 2, the decoder 200 includes an entropy decoding unit210, a dequantization unit 220, an inverse transform unit 225, afiltering unit 230, a DPB unit 240, an inter-prediction unit 250, and anintra-prediction unit 255. Furthermore, a reconstructed signal output bythe decoder 200 may be played back through a display 260.

The decoder 200 receives a signal output by the encoder 100 of FIG. 1.The received signal is subject to entropy decoding through the entropydecoding unit 210. The dequantization unit 220 obtains a transformcoefficient from the entropy-decoded signal using quantization step sizeinformation. The inverse transform unit 225 obtains a residual signal byperforming inverse transform on the transform coefficient. Thereconstructed signal is generated by adding the obtained residual signalto a prediction signal output by the inter-prediction unit 250 or theintra-prediction unit 255.

The filtering unit 230 applies filtering to the reconstructed signal andoutputs the filtered signal to a playback device or sends the filteredsignal to the DPB unit 240. The filtered signal sent to the DPB unit 240may be used as a reference frame in the inter-prediction unit 250. Inthis specification, embodiments illustrated in filtering unit 140 of theencoder may be likewise applied to the filtering unit 230 of thedecoder.

FIG. 3 schematically illustrates location relationships between sampleswithin a block used when filtering is performed in accordance with anembodiment to which the present invention is applied.

Deblocking filtering in video coding may be chiefly divided into ananalysis stage and a filtering stage.

In the analysis stage, the coding artifacts of a decoded picture may bemodeled through a parameter set that is related to a 1D representationof a finite tap-length and artifacts-feature estimation. For example, asillustrated in FIG. 3, four samples of each of a left block and a rightblock on the basis of the vertical boundary of the block may be used.Furthermore, determination in a sample data processing and artifactsmodeling stage is performed based on a specific deblocking type and theparameter of a deblocking filter.

Blocking artifacts may occur in both a vertical edge and a horizontaledge. Thus, filtering may be first performed on a vertical edge thatbelongs to the block boundary of a picture that is now reconstructed ina horizontal direction, and filtering may be then performed on ahorizontal edge that belongs to the block boundary of the picture thatis now reconstructed. In this specification, embodiments in whichfiltering is horizontally performed on a vertical edge have beenillustrated, but the embodiments may be likewise applied to thefiltering of a horizontal edge. Order of such filtering is also notlimited to the above examples.

Referring to FIG. 3, sample positions on the left side may be defined asp0, p1, p2, and p3, and sample positions on the right side may bedefined as q0, q1, q2, and q3 on the basis of a vertical boundary.Furthermore, a sub-script denotes the row identifier of a sample. Forexample, the sample positions of a left block P in a first row may bedefined as p0 ₀, p1 ₀, p2 ₀, and p3 ₀, and the sample positions of aright block Q may be defined as q0 ₀, q1 ₀, q2 ₀, and q3 ₀ (310).Likewise, the sample positions of the left block P in a fourth row maybe defined as p0 ₃, p1 ₃, p2 ₃, and p3 ₃, and the sample positions ofthe right block Q may be defined as q0 ₃, q1 ₃, q2 ₃, and q3 ₃ (320).

The analysis stage is required to use various coding parameters, such ascoding mode, a coded residual signal, motion information, and an actualsample at a block boundary.

The analysis stage can be considered as a complicated part of thedeblocking filtering and requires access to various coding parameterssuch as coding modes (intra, or inter), coded residuals, motioninformation (motion vectors and reference index) as well as actualsamples at the block boundary.

As some of such parameters are present at a block level, complexity offiltering would follow block partitioning used to code a currentpicture. An example of such processing may include the derivation ofBoundary Strength (BS).

A less complex video signal would be represented with a larger blocksize and therefore less parameters to be fetched from memory andprocessed for deblocking. For example, simple content is likely to becoded with large blocks and complex content is likely to be coded with alarge number of small blocks.

Accordingly, complexity of filtering can be reduced by applying locallyadaptive sampling based on information about the partitioning and/orcoding mode of each block rather than applying a fixed manner to all theblocks when filtering is performed.

FIG. 4 is a schematic internal block diagram of the filtering unit forperforming locally adaptive sampling in accordance with an embodiment towhich the present invention is applied.

The filtering unit 140, 230 includes a picture property checking unit410, an adaptive sampling unit 420, and a filtering execution unit 430.

The filtering unit 140, 230 may first perform filtering on verticaledges for each macro block in a horizontal direction and then performfiltering on horizontal edges in a vertical direction. In someembodiments, the filtering unit 140, 230 may perform filtering on thevertical edges of all the block boundaries of a picture that is nowreconstructed in a picture unit and perform filtering on all thehorizontal edges of all the block boundaries of the picture, but thepresent invention is not limited thereto.

First, the picture property checking unit 410 may check the propertyinformation of a decoded picture. For example, the property informationof the decoded picture may include a block size, a picture parameter,etc. The picture parameter may be information that is included in a bitstream and transmitted or may be information derived from by thedecoder. However, the present invention is not limited thereto. Forexample, the property information of decoded data can be defined atdifferent levels, e.g. SPS (Sequence Parameter Set), PPS (PictureParameter Set), slice or LCU (Largest Coding Unit), etc. Accordingly, anadaptive sampling rate and/or an adaptive sampling pattern can bedetermined based on parameters being signaled or derived at differentlevels.

The adaptive sampling unit 420 may determine an adaptive sampling rateand/or an adaptive sampling pattern based on the property informationreceived from the picture property checking unit 410.

For example, the adaptive sampling unit 420 may determine an adaptivesampling rate and/or an adaptive sampling pattern based on the size of aleft block and the size of a right block on the basis of a blockboundary. For example, if the size of a left block is the same as thatof a right block, the adaptive sampling unit 420 may identically applythe adaptive sampling rate and/or the adaptive sampling pattern to theleft block and the right block.

For another example, if the size of the left block is different fromthat of the right block, the adaptive sampling unit 420 may differentlyapply the adaptive sampling rate and/or the adaptive sampling pattern tothe left block and the right block. For example, the adaptive samplingunit 420 may apply low frequency sampling on a block having a largerblock size or may apply reduced sampling on a block having a largerblock size.

Meanwhile, a location of a sample to be filtered may be addressed by arelative location for the left-most pixel of a right block on the basisof a block boundary. Furthermore, the adaptive sampling rate may bedetermined by scaling an offset with a rate conversion value, and therate conversion value may be differently applied to the left block andthe right block on the basis of the block boundary.

Furthermore, the location of the sample to be filtered may beadditionally adjusted based on a sub-rate conversion offset value.

The filtering execution unit 430 may perform filtering by applying anadaptive sampling rate and/or an adaptive sampling pattern determined bythe adaptive sampling unit 420. That is, the filtering execution unit430 may perform filtering using a sample to which the adaptive samplingrate and/or the adaptive sampling pattern has been applied.

FIGS. 5 and 6 are embodiments to which the present invention is applied,wherein FIG. 5 is a flowchart illustrating a method of performinglocally adaptive sampling and FIG. 6 is a flowchart illustrating amethod of performing filtering using locally adaptive sampling.

In an embodiment of the present invention, in order to determine whetheror not to perform filtering, first, the property information of adecoded picture may be checked at step S510.

For example, the property information of the decoded picture may includeat least one of a coding block size, a prediction block size, atransform block size, a partitioned block size, coding mode, and acoding parameter.

An adaptive sampling rate and/or an adaptive sampling pattern may bedetermined based on the checked property information of the decodedpicture at step S520.

For example, the adaptive sampling rate and/or the adaptive samplingpattern may be determined based on the size of a left block and the sizeof a right block on the basis of a block boundary. If the size of theleft block is the same as that of the right block, the adaptive samplingrate and/or the adaptive sampling pattern may be identically applied tothe left block and the right block.

If the size of the left block is different from that of the right block,the adaptive sampling rate and/or the adaptive sampling pattern may bedifferently applied to the left block and the right block. For example,low frequency sampling may be applied to a block having a larger blocksize, or reduced sampling may be applied to a block having a largerblock size.

After the adaptive sampling rate and/or the adaptive sampling patternare determined as described above, filtering may be performed using asample to which the adaptive sampling rate and/or the adaptive samplingpattern has been applied at step S530.

For example, referring to FIG. 6, when a sample to be filtered isdetermined by applying an adaptive sampling rate and/or an adaptivesampling pattern at step S610, an offset value may be calculated byapplying a finite impulse response (FIR) filter to the sample at stepS620. In this case, a sample value may be replaced with the calculatedoffset value at step S630.

Furthermore, the value of the filtered sample may be obtained at stepS650 by applying a clipping function to the calculated offset value atstep S640.

FIGS. 7 to 9 are diagrams illustrating that an adaptive sampling rate isdetermined based on the property information of a decoded picture inaccordance with embodiments to which the present invention is applied.

The present invention proposes a local adaptive sampling method in afiltering stage. The local adaptive sampling method may be determinedbased on the property information of a decoded picture. For example, anadaptive sampling rate and/or an adaptive sampling pattern may bedetermined depending on the size of a left block and the size of a rightblock on the basis of a block boundary. In this case, the adaptivesampling rate and/or the adaptive sampling pattern may be defined by atleast one of the number of samples and the interval between samples.

In the embodiment of FIG. 7, if the size of a left block P is the sameas that of a right block Q, the adaptive sampling rate and/or theadaptive sampling pattern may be identically applied to the left block Pand the right block Q.

Referring to FIG. 7, if the sizes of neighboring blocks on the basis ofa block boundary are the same, the number of samples used within theleft block P may be the same as that used within the right block Q.Furthermore, the interval between samples used within the left block Pmay be the same as the interval between samples used within the rightblock Q.

In the embodiments of FIGS. 8 and 9, if the size of a left block P isnot the same as that of a right block Q, the adaptive sampling rateand/or the adaptive sampling pattern may be differently applied to theleft block P and the right block Q.

Referring to FIG. 8, if the size of the left block P is larger than thatof the right block Q, relative low frequency sampling may be applied tothe left block P. In this case, the low frequency sampling may bedetermined based on complexity estimates of video signal undergoingcoding. And, the low frequency sampling may be randomly determined ormay be applied based on a relative size of a block.

For example, in FIG. 8, if the size of the left block P is 32×32 and thesize of the right block Q is 16×16, the interval between samples of theleft block P to be filtered may be twice the interval between samples ofthe right block Q to be filtered, and the number of samples of the leftblock P to be filtered and the number of samples of the right block Q tobe filtered may be the same, that is, 4.

Referring to FIG. 9, if the size of the left block P is larger than thatof the right block Q, relatively reduced sampling may be applied to theleft block P. In this case, the reduced sampling may be determined basedon complexity estimates of video signal undergoing coding. And, thereduced sampling may be randomly determined or may be applied based on arelative size of a block.

For example, in FIG. 9, if the size of the left block P is 32×32 and thesize of the right block Q is 16×16, the number of samples of the leftblock P to be filtered may be ½ of the number of samples of the rightblock Q to be filtered, and the interval between the samples of the leftblock P to be filtered may be four times the interval between thesamples of the right block Q to be filtered. That is, the number ofsamples of the left block P to be filtered may be two, that is, p0 andp2, and the number of samples of the right block Q to be filtered may befour, that is, q0, q1, q2, and q3.

FIG. 10 is a diagram illustrating a method of performing filtering usinglocally adaptive sampling in accordance with an embodiment to which thepresent invention is applied.

Referring to FIG. 10, it is assumed that sample positions on the leftside are p0, p1, p2, and p3 and sample positions on the right side areq0, q1, q2, and q3 on the basis of a vertical boundary, and d0, d1, d2,d3, d4, and d5 are offset values that replace pixel values placed atcorresponding sample positions. For example, in FIG. 10, the offsetvalues d0, d1, and d2 may replace respective pixel values I(p2), I(p1),and I(p0) placed at the sample positions p2, p1, p0 of the left block P,and the offset values d3, d4, and d5 may replace respective pixel valuesI(q0), I(q1), and I(q2) placed at the sample positions q0, q1, and q2 ofthe right block Q (1000).

The offset values d0, d1, d2, d3, d4, and d5 may be calculated byapplying a finite impulse response (FIR) filter, such as that of MathFigure 1 below.

d0=((2*I(p3)+3*I(p2)+I(p1)+I(p0)−I(q0)+4)>>3);

d1=((I(p2)+I(p2)+I(p3)+I(p4)+2))>>2);

d2=((I(p2)+2*I(p1)+2*I(p0)+2*I(q0)+I(q1)+4)>>3);

d3=((I(p1)+2*I(p0)+2*I(q0)+2*I(q1)+I(q2)+4)>>3);

d4=((I(p0)+I(q0)+I(q1)+I(q2)+2)>>2);

d5=((I(p0)+I(q0)+I(q1)+3*I(q2)+2*I(q3)+4)>>3);  [Math Figure 1]

The finite impulse response (FIR) filter applied in Math Figure 1 isonly an embodiment, and the present invention is not limited thereto.

If the offset values d0˜d5 are calculated using Math Figure 1, aclipping function, such as that of Math Figure 2, may be applied inorder to replace corresponding sample positions p2˜p0, q0˜q2.

I(p2)=Clip3(A,A′,d0);

I(p1)=Clip3(B,B′,d1);

I(p0)=Clip3(C,C′,d2);

I(q0)=Clip3(D,D′,d3);

I(q1)=Clip3(E,E′,d4);

I(q2)=Clip3(F,F′,d5);  [Math Figure 2]

In this case, A-F and A′˜F′ are indicative of the upper and lowerdynamical range boundaries of the samples p3˜q3. That is, the offsetvalues d0˜d5 converge within a range that include A and A′, B and B′, .. . , F and F′, respectively. In such a case, the ranges of samplevalues replaced by filtering are limited to the upper limit values A˜Fand the lower limit values A′˜F′. The upper limit values A˜F and thelower limit values A′˜F′ may be determined based on the quantizationparameters of the left block P and the right block Q. For example, ifthe quantization parameter has a great value, ranges determined by theupper limit values A˜F and the lower limit values A′˜F′ may beincreased.

Meanwhile, the sample positions p3˜q3 may be determined by relativelocations on the basis of the block boundary. For example, the samplepositions p3˜q3 may be addressed by a relative location for theleft-most pixel of the right block Q on the basis of the block boundary.The spatial sample positions p3˜q3 may be sequentially identified usingthe left-most pixel of the right block Q as a reference location.

Assuming a row of a decoded picture is represented as “piSrc”, theleft-most pixel of the right block Q is given as “piSrc[0]”. If anoffset indicative of the original pixel grid is 1, the sample positionsp3˜q3 may be given as follows.

p3=piSrc[−Offset*4];

p2=piSrc[−Offset*3];

p1=piSrc[−Offset*2];

p0=piSrc[−Offset];

q0=piSrc[0];

q1=piSrc[Offset];

q2=piSrc[Offset*2];

q3=piSrc[Offset*3];  [Math Figure 3]

In some embodiments of the present invention, the sample positions p3˜q3may be addressed at a locally adaptive sampling rate. For example, thelocally adaptive sampling rate may be performed by scaling the offsetvalue with a rate conversion (RP) value as in Math Figure 4 below.

p3=piSrc[−Offset*RP*4];

p2=piSrc[−Offset*RP*3];

p1=piSrc[−Offset*RP*2];

p0=piSrc[−Offset*RP];

q0=piSrc[0];

q1=piSrc[Offset*RP];

q2=piSrc[Offset*RP*2];

q3=piSrc[Offset*RP*3];  [Math Figure 4]

In Math Figure 4, RP denotes a rate conversion value.

In some embodiments of the present invention, different RP values may beapplied to the left block P and the right block Q on the basis of theblock boundary. For example, a rate conversion value applied to the leftblock P may be defined as a RPL (left) scaling factor, and a rateconversion value applied to the right block Q may be defined as an RPR(right) scaling factor. In this case, the sample positions p3˜q3 may begiven as in Math Figure 5 below.

p3=piSrc[−Offset*RPL*4];

p2=piSrc[−Offset*RPL*3];

p1=piSrc[−Offset*RPL*2];

p0=piSrc[−Offset*RPL];

q0=piSrc[0];

q1=piSrc[Offset*RPR];

q2=piSrc[Offset*RPR*2];

q3=piSrc[Offset*RPR*3];  [Math Figure 5]

In accordance with another embodiment of the present invention, a sampleposition may be additionally refined based on a sub-rate conversionoffset value. For example, a sample position may be additionallyadjusted precisely by scaling the offset value with a rate conversionvalue and then adding a sub-rate conversion offset value. In this case,the sample positions p3˜q3 may be given as in Math Figure 6 below.

p3=piSrc[−Offset*RPL*4+subRPL];

p2=piSrc[−Offset*RPL*3+subRPL];

p1=piSrc[−Offset*RPL*2+subRPL];

p0=piSrc[−Offset*RPL+subRPL];

q0=piSrc[0+subRPR];

q1=piSrc[Offset*RPR+subRPR];

q2=piSrc[Offset*RPR*2+subRPR];

q3=piSrc[Offset*RPR*3+subRPR];  [Math Figure 6]

In Math Figure 6, “subRPL” is indicative of a sub-rate conversion offsetvalue applied to the left block P, and “subRPR” is indicative of asub-rate conversion offset value applied to the right block Q.

FIG. 11 illustrates various examples in which a sampling is performedusing a rate conversion value and a sub-rate conversion offset value inaccordance with an embodiment to which the present invention is applied.

FIG. 11(a) illustrates an example in which RPL=2, subRPL=0, RPR=2, andsubRPR=0. That is, the present embodiment corresponds to an example inwhich all the sub-rate conversion offset values are 0 and the same rateconversion value, that is, 2, has been applied to the left block P andthe right block Q.

FIG. 11 (b) illustrates an example in which RPL=2, subRPL=0, RPR=2, andsubRPR=1. That is, the present embodiment corresponds to an example inwhich a sub-rate conversion offset value 1 has been applied to only theright block Q and the same rate conversion value, that is, 2, has beenapplied to the left block P and the right block Q. When the example ofFIG. 11(b) is compared with the example of FIG. 11(a), it may be seenthat the locations of all the samples in the right block Q have beenshifted to the right by 1 pixel.

FIG. 11(c) illustrates an example in which RPL=1, subRPL=0, RPR=2, andsubRPR=1. The present embodiment corresponds to an example in which arate conversion value and a sub-rate conversion offset value aredifferently applied to the left block P and the right block Q. That is,a rate conversion value applied to the right block Q is two times a rateconversion value applied to the left block P, and a sub-rate conversionoffset value 1 has been applied to only the right block Q. When theexample of FIG. 11(c) is compared with the example of FIG. 11(b), it maybe seen that the locations of all the samples in the left block P havebeen shifted to the block boundary by ½.

In yet another embodiment of the present invention, the values RPR, RPL,subRPL, and subRPR or the subset values of them may be determined by theencoder and the decoder from local parameters within spatio-temporallyneighboring decoded pictures. For example, the local parameters mayinclude block partitioning, coding modes, motion information, atransform type, decoded picture sample values, and other availablepriori information.

In yet another embodiment of the present invention, the values RPR, RPL,subRPL, and subRPR or the subset values of them may be determined may bedetermined by the encoder and the decoder from local parameters withinspatio-temporally neighboring decoded pictures. For example, the localparameters may include block partitioning, coding modes, motioninformation, a transform type, decoded picture sample values, otheravailable priori information, and a signal, such as syntax elements(i.e., a block partitioning level) or side information (e.g., SEI in thecase of post-processing).

In some embodiments of the present invention, update values d0˜d5 may beapplied to decoded picture samples located based on a rate conversionvalue and a sub-rate conversion offset value as in Math Figure 7 below.

piSrc[−Offset*RPL*3+subRPL]=d0;

piSrc[−Offset*RPL*2+subRPL]=d1;

piSrc[−Offset*RPL+subRPL]=d2;

piSrc[0+subRPL]=d3;

piSrc[Offset*RPL+subRPR]=d4;

piSrc[Offset*RPL*2+subRPR]=d5;  [Math Figure 7]

In some embodiments of the present invention, if the value RPL, RPR isgreater than 1, samples located at a sub-integer pixel grid may beupdated through the interpolation of two nearest update values. Forexample, if RPL=RPR=2 and linear interpolation is used, Math Figure 8may be obtained.

piSrc[−Offset*RPL*3]=d0;

piSrc[−Offset*RPL*3]=d0;

piSrc[−Offset*RPL*3+1]=(d0+d1)/2;

piSrc[−Offset*RPL*2]=d1;

piSrc[−Offset*RPL*2+1]=(d1+d2)/2;

piSrc[−Offset*RPL]=d2;

piSrc[−Offset*RPL+1]=(3*d2+d3)/4;

piSrc[0]=(d2+3*d3)/4;

piSrc[0+1]=d3;

piSrc[Offset*RPL]=(d3+d4)/2:

piSrc[Offset*RPL+1]=d4;

piSrc[Offset*RPL*2]=(d4+d5)/2;

piSrc[Offset*RPL*2+1]=d5;  [Math Figure 8]

In yet another embodiment of the present invention, other interpolationmethods may be used to produce the sample values located at asub-integer pixel grid. Not-limiting examples may include a quadratic,cubic, higher order, spline, transform-based interpolation, non-linearinterpolation, and adaptive interpolation method.

The update values d0˜d5 as well as the sample values of such samples maybe calculated by stretching, interpolation, and extrapolating pulseresponses provided by Math Figure 1 and then applying a windowingfunction to them.

In yet another embodiment of the present invention, an adaptive pulseresponse function may be derived on the decoder or encoder side througha specified process or may be signaled using syntax elements within abit stream or side information.

In some embodiments of the present invention, if a great block size isused, in deblocking, an analysis of block border conditions using asub-sampled version of boundary samples may be performed. Alternatively,deblocking may use filtering of a different tap length in order toincorporate information on the local complexity of a video signal.

In some embodiments of the present invention, interpolation filtershaving variable sampling rates may be applied depending on thecomplexity estimates of undergoing coding.

In some embodiments of the present invention, adaptive filters, aSample-Adaptive Offset (SAO) filter, or transform may be applied withvarious sampling rates and/or tap lengths/adaptive filter coefficientsbased on the complexity estimates of undergoing coding.

In another embodiment to which the present invention is applied, thefiltering unit 140, 230 may determine a Boundary Strength (BS) valuebased on at least one of the size, coding mode, and coding parameter ofa block that neighbors a block boundary. Whether or not to performfiltering may be determined based on the BS value. For example, in orderto determine whether or not to perform filtering, a change of a samplevalue may be measured based on the samples in the first row (see 310 ofFIG. 3) and the samples in the fourth row (see 320 of FIG. 3). In such acase, the locally adaptive sampling method described in thisspecification may be applied.

In addition, the filtering unit 140, 230 may calculate another variablevalue using the quantization parameter values of blocks that neighborthe block boundary and determine whether or not to perform filteringbased on another variable value. If conditions to which filtering isapplied are satisfied, the filtering unit 140, 230 may select a filtertype to be applied to the block boundary.

If filtering is performed on horizontal edges, it may be performed in arow unit. Furthermore, filtering may be performed on a specific numberof samples on the basis of a block boundary. For example, if strongfiltering is performed, three samples within a block may be used. Ifweak filtering is performed, two samples within a block may be used.Even in such a case, the locally adaptive sampling method described inthis specification may be applied.

As described above, the decoder and the encoder to which the presentinvention is applied may be included in a multimedia broadcastingtransmission/reception apparatus, a mobile communication terminal, ahome cinema video apparatus, a digital cinema video apparatus, asurveillance camera, a video chatting apparatus, a real-timecommunication apparatus, such as video communication, a mobile streamingapparatus, a storage medium, a camcorder, a VoD service providingapparatus, an Internet streaming service providing apparatus, athree-dimensional (3D) video apparatus, a teleconference videoapparatus, and a medical video apparatus and may be used to code videosignals and data signals.

Furthermore, the decoding/encoding method to which the present inventionis applied may be produced in the form of a program that is to beexecuted by a computer and may be stored in a computer-readablerecording medium. Multimedia data having a data structure according tothe present invention may also be stored in computer-readable recordingmedia. The computer-readable recording media include all types ofstorage devices in which data readable by a computer system is stored.The computer-readable recording media may include a BD, a USB, ROM, RAM,CD-ROM, a magnetic tape, a floppy disk, and an optical data storagedevice, for example. Furthermore, the computer-readable recording mediaincludes media implemented in the form of carrier waves (e.g.,transmission through the Internet). Furthermore, a bit stream generatedby the encoding method may be stored in a computer-readable recordingmedium or may be transmitted over wired/wireless communication networks.

INDUSTRIAL APPLICABILITY

The exemplary embodiments of the present invention have been disclosedfor illustrative purposes, and those skilled in the art may improve,change, replace, or add various other embodiments within the technicalspirit and scope of the present invention disclosed in the attachedclaims.

1. A method of performing a deblocking filtering on a video signal,comprising: determining an adaptive sampling rate based on propertyinformation of a decoded picture; and performing the deblockingfiltering using samples to which the adaptive sampling rate has beenapplied, wherein the property information of the decoded pictureincludes at least one of a block size and a picture parameter.
 2. Themethod of claim 1, wherein if a size of a left block is identical with asize of a right block based on a block boundary, the adaptive samplingrate is identically applied to the left block and the right block. 3.The method of claim 1, wherein if a size of a left block is differentfrom a size of a right block based on a block boundary, the adaptivesampling rate is differently applied to the left block and the rightblock, and low frequency sampling or reduced sampling is applied to ablock having a larger size.
 4. The method of claim 1, wherein thesamples are addressed by a relative location for a left-most pixel of aright block based on a block boundary.
 5. The method of claim 4, whereinthe adaptive sampling rate is determined by scaling an offset value witha rate conversion value.
 6. The method of claim 5, wherein the rateconversion value is differently applied to a left block and the rightblock based on the block boundary.
 7. The method of claim 5, wherein alocation of the samples is additionally adjusted based on a sub-rateconversion offset value.
 8. The method of claim 1, wherein a step ofperforming the deblocking filtering comprises: determining samples to befiltered based on the adaptive sampling rate; calculating offset valuesby applying a finite impulse response (FIR) filter to the samples; andobtaining filtered sample values by applying a clipping function to theoffset values.
 9. The method of claim 1, wherein the picture parameteris obtained from coded bit stream or derived from decoded picture. 10.An apparatus for performing a deblocking filtering on a video signal,comprising: a picture property checking unit configured to checkproperty information of a decoded picture; an adaptive sampling unitconfigured to determine an adaptive sampling rate based on the propertyinformation of the decoded picture; and a filtering execution unitconfigured to perform the deblocking filtering using samples to whichthe adaptive sampling rate has been applied, wherein the propertyinformation of the decoded picture includes at least one of a block sizeand a picture parameter.
 11. The apparatus of claim 10, wherein if asize of a left block is identical with a size of a right block based ona block boundary, the adaptive sampling rate is identically applied tothe left block and the right block.
 12. The apparatus of claim 10,wherein if a size of a left block is different from a size of a rightblock based on a block boundary, the adaptive sampling rate isdifferently applied to the left block and the right block, and lowfrequency sampling or reduced sampling is applied to a block having alarger size.
 13. The apparatus of claim 10, wherein the samples areaddressed by a relative location for a left-most pixel of a right blockbased on a block boundary.
 14. The apparatus of claim 13, wherein theadaptive sampling rate is determined by scaling an offset with a rateconversion value.
 15. The apparatus of claim 14, wherein the rateconversion value is differently applied to a left block and the rightblock based on the block boundary.
 16. The apparatus of claim 14,wherein a location of the samples is additionally adjusted based on asub-rate conversion offset value.
 17. The apparatus of claim 10, whereinthe filtering execution unit is further configured to determine samplesto be filtered based on the adaptive sampling rate, calculate offsetvalues by applying a finite impulse response (FIR) filter to thesamples, and obtain filtered sample values by applying a clippingfunction to the offset values.
 18. The apparatus of claim 10, whereinthe picture parameter is obtained from coded bit stream or derived fromdecoded picture.